An Evaluation of Mouse and Keyboard Interaction Indicators towards Non-intrusive and Low Cost Affective Modeling in an Educational Context

AbstractIn this paper we propose a series of indicators, which derive from user's interactions with mouse and keyboard. The goal is to evaluate their use in identifying affective states and behavior changes in an e-learning platform by means of non-intrusive and low cost methods. The approach we have followed study user's interactions regardless of the task being performed and its presentation, aiming at finding a solution applicable in any domain. In particular, mouse movements and clicks, as well as keystrokes were recorded during a math problem solving activity where users involved in the experiment had not only to score their degree of valence (i.e., pleasure versus displeasure) and arousal (i.e., high activation versus low activation) of their affective states after each problem by using the Self-Assessment-Manikin scale, but also type a description of their own feelings. By using that affective labeling, we evaluated the information provided by these different indicators processed from the original user's interactions logs. In total, we computed 42 keyboard indicators and 96 mouse indicators

Towards Emotion Detection in Educational Scenarios from Facial Expressions and Body Movements through Multimodal Approaches

We report current findings when considering video recordings of facial expressions and body movements to provide affective personalized support in an educational context from an enriched multimodal emotion detection approach. In particular, we describe an annotation methodology to tag facial expression and body movements that conform to changes in the affective states of learners while dealing with cognitive tasks in a learning process. The ultimate goal is to combine these annotations with additional affective information collected during experimental learning sessions from different sources such as qualitative, self-reported, physiological, and behavioral information. These data altogether are to train data mining algorithms that serve to automatically identify changes in the learners’ affective states when dealing with cognitive tasks which help to provide emotional personalized support